Zamir A R, Sax A, Shen W, et al. Taskonomy: Disentangling Task Transfer Learning[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 3712-3722.
1. Overview
1.1. Motivation
- relationships among different visual tasks
In this paper
- proposes a fully computational approach for modeling the structure of space of visual task
- exploit it to reduce the demand for labeled data
1.2. Related Works
- self-supervised learning
- unsupervised learning
- meta-learning
- multi-task learning
- domain adaption
2. Methods
2.1. Definitions
- γ. limited supervision budget
- T (target). set of task want to solve
- S (source). set of task can be trained
- V=T∪S. task dictionary
- T∩S. task want to solve but can play as source
- T-T∩S. task can not trained (target only)
S-T∩S. (source only)
edge. between a group of source and target tasks, represent feasible case
- weight. prediction of its performance
- use these edges to estimate the globally optimal transfer policy to solve T
Four Steps
2.2. Stage I: Task-Specific Modeling
- Encoder-Decoder
2.3. Stage II: Transfer Modeling
learn a readout function
E_s. encoder
- D_Θ. readout function
- f_t. gt of task t for image I.
- the performance of D_{s→ t} is a useful metric as task affinity
- use shallow fully convolutional network
- combinatorial explosion of higher-oder transfer. using beam search to sample
2.4. Stage III: Ordinal Normalization using Analytic Hierarchy Process (AHP)
- aggregating the raw losses/evaluations L_{s→ t} from transfer function into a matrix is problem. vastly different scale and live in different spaces
- naive solution of linearly rescale each fail
2.5. Step IV: Computing the Global Texonomy
edge (hypergraph)
Boolean Integer Programming (BIP) to find global policy
2.6. Constrains
- each target task has exactly one transfer in
- supervision budget is not exceeded
